Modeling Synergistic Relationships Between Words and Images

Texts and images provide alternative, yet orthogonal views of the same underlying cognitive concept. By uncovering synergistic, semantic relationships that exist between words and images, I am working to develop novel techniques that can help improve tasks in natural language processing, as well as effective models for text-to-image synthesis, image retrieval, and automatic image annotation. Specifically, in my dissertation, I will explore the interoperability of features between language and vision tasks. In the first part, I will show how it is possible to apply features generated using evidence gathered from text corpora to solve the image annotation problem in computer vision, without the use of any visual information. In the second part, I will address research in the reverse direction, and show how visual cues can be used to improve tasks in natural language processing. Importantly, I propose a novel metric to estimate the similarity of words by comparing the visual similarity of concepts invoked by these words, and show that it can be used further to advance the state-of-the-art methods that employ corpus-based and knowledge-based semantic similarity measures. Finally, I attempt to construct a joint semantic space connecting words with images, and synthesize an evaluation framework to quantify cross-modal semantic relationships that exist between arbitrary pairs of words and images. I study the effectiveness of unsupervised, corpus-based approaches to automatically derive the semantic relatedness between words and images, and perform empirical evaluations by measuring its correlation with human annotators

Towards Communicating Simple Sentence using Pictorial Representations

Language can sometimes be an impediment in communication. Whether we are talking about people who speak different languages, students who are learning a new language, or people with language disorders, the understanding of linguistic representations in a given language requires a certain amount of knowledge that not everybody has. In this thesis, we propose "translation through pictures" as a means for conveying simple pieces of information across language barriers, and describe a system that can automatically generate pictorial representations for simple sentences. Comparative experiments conducted on visual and linguistic representations of information show that a considerable amount of understanding can be achieved through pictorial descriptions, with results within a comparable range of those obtained with current machine translation techniques. Moreover, a user study conducted around the pictorial translation system reveals that users found the system to generally produce correct word/image associations, and rate the system as interactive and intelligent

Text Mining for Automatic Image Tagging

This paper introduces several extractive approaches for automatic image tagging, relying exclusively on information mined from texts. Through evaluations on two datasets, the authors show that their methods exceed competitive baselines by a large margin, and compare favorably with the state-of-the-art that uses both textual and image features

Detection of Gravitational Wave - An Application of Relativistic Quantum Information Theory

We show that a passing gravitational wave may influence the spin entropy and spin negativity of a system of $N$ massive spin-1/2 particles, in a way that is characteristic of the radiation. We establish the specific conditions under which this effect may be nonzero. The change in spin entropy and negativity, however, is extremely small. Here, we propose and show that this effect may be amplified through entanglement swapping. Relativistic quantum information theory may have a contribution towards the detection of gravitational wave.Comment: 9 page

Impact of within-row plant spacing and fixed fruit setting on yield and quality of rockmelon fruit cultivated by drip irrigation in a greenhouse

Experiments were conducted in a ventilated greenhouse located in a commercial farm (2°56′N, 101°54′E) at Broga, Malaysia from 2014 to 2015. The objective of this study was to evaluate the interaction between within-row plant spacing and fruit-set position on the yield and quality of 'Glamour' rockmelon fruit. Rockmelon was grown at 1.57 m between rows with 0.15 m, 0.30 m, and 0.60 m of within-row plant spacing. The plants were fixed to set fruit at T1 (one fruit, 1-7 lateral branches), T2 (one fruit, 8-14 lateral branches), T3 (two fruits, 1-7 lateral branches), and T4 (two fruits, 8-14 lateral branches). As within-row plant spacing decreased from 0.30 m to 0.15 m, yield per ha was increased by 58.1%, and a 3.7-fold increase in yield was seen from 0.60-m to 0.15-m spacing. Total soluble solids (11.96°Brix) and total carbohydrate content (86.60 g GE 100 g DW) were highest at a plant spacing of 0.60 m. Our results demonstrated that the interaction between 0.30-m within-row plant spacing with two fruits set at 8-14 lateral branches showed the most commercially viable outcome in fresh fruit weight as these conditions produced two fruits with an average fruit weight of 2.20 kg per fruit

Bioactivity, physical and chemical properties of MTA mixed with propylene glycol

AbstractObjective To investigate the physical (setting time, hardness, flowability, microstructure) and chemical (pH change, calcium release, crystallinity) properties and the biological outcomes (cell survival and differentiation) of mineral trioxide aggregate (MTA) mixed using different proportions of propylene glycol (PG) and water.Material and Methods White MTA was mixed with different water/PG ratios (100/0, 80/20 and 50/50). Composition (XRD), microstructure (SEM), setting time (ASTM C266-13), flowability (ANSI/ADA 57-2000), Knoop hardness (100 g/10 s) and chemical characteristics (pH change and Ca2+ release for 7 days) were evaluated. Cell proliferation, osteo/odontoblastic gene expression and mineralization induced by MTA mixed with PG were evaluated. MTA discs (5 mm in diameter, 2 mm thick) were prepared and soaked in culture medium for 7 days. Next, the discs were removed and the medium used to culture dental pulp stem cells (DPSC) for 28 days. Cells survival was evaluated using MTS assay (24, 72 and 120 h) and differentiation with RT-PCR (ALP, OCN, Runx2, DSPP and MEPE) and alizarin red staining (7 and 14 days). Data were analysed using one-way ANOVA and Tukey’s post-hoc analysis (a=0.05).Results The addition of PG significantly increased setting time, flowability and Ca2+ release, but it compromised the hardness of the material. SEM showed that 50/50 group resulted porous material after setting due to the incomplete setting reaction, as shown by XRD analysis. The addition of PG (80/20 and 50/50) was not capable to improve cell proliferation or to enhance gene expression, and mineralized deposition of DPSC after 7 and 14 days as compared to the 100/0.Conclusion Except for flowability, the addition of PG did not promote further improvements on the chemical and physical properties evaluated, and it was not capable of enhancing the bioactivity of the MTA

Effects of a robot-assisted training of grasp and pronation/supination in chronic stroke: a pilot study

<p>Abstract</p> <p>Background</p> <p>Rehabilitation of hand function is challenging, and only few studies have investigated robot-assisted rehabilitation focusing on distal joints of the upper limb. This paper investigates the feasibility of using the <it>HapticKnob</it>, a table-top end-effector device, for robot-assisted rehabilitation of grasping and forearm pronation/supination, two important functions for activities of daily living involving the hand, and which are often impaired in chronic stroke patients. It evaluates the effectiveness of this device for improving hand function and the transfer of improvement to arm function.</p> <p>Methods</p> <p>A single group of fifteen chronic stroke patients with impaired arm and hand functions (Fugl-Meyer motor assessment scale (FM) 10-45/66) participated in a 6-week 3-hours/week rehabilitation program with the <it>HapticKnob</it>. Outcome measures consisted primarily of the FM and Motricity Index (MI) and their respective subsections related to distal and proximal arm function, and were assessed at the beginning, end of treatment and in a 6-weeks follow-up.</p> <p>Results</p> <p>Thirteen subjects successfully completed robot-assisted therapy, with significantly improved hand and arm motor functions, demonstrated by an average 3.00 points increase on the FM and 4.55 on the MI at the completion of the therapy (4.85 FM and 6.84 MI six weeks post-therapy). Improvements were observed both in distal and proximal components of the clinical scales at the completion of the study (2.00 FM wrist/hand, 2.55 FM shoulder/elbow, 2.23 MI hand and 4.23 MI shoulder/elbow). In addition, improvements in hand function were observed, as measured by the Motor Assessment Scale, grip force, and a decrease in arm muscle spasticity. These results were confirmed by motion data collected by the robot.</p> <p>Conclusions</p> <p>The results of this study show the feasibility of this robot-assisted therapy with patients presenting a large range of impairment levels. A significant homogeneous improvement in both hand and arm function was observed, which was maintained 6 weeks after end of the therapy.</p